Search for: All records

This paper considers an infinite-horizon Markov decision process (MDP) that allows for general nonexponential discount functions in both discrete and continuous time. Because of the inherent time inconsistency, we look for a randomized equilibrium policy (i.e., relaxed equilibrium) in an intrapersonal game between an agent’s current and future selves. When we modify the MDP by entropy regularization, a relaxed equilibrium is shown to exist by a nontrivial entropy estimate. As the degree of regularization diminishes, the entropy-regularized MDPs approximate the original MDP, which gives the general existence of a relaxed equilibrium in the limit by weak convergence arguments. As opposed to prior studies that consider only deterministic policies, our existence of an equilibrium does not require any convexity (or concavity) of the controlled transition probabilities and reward function. Interestingly, this benefit of considering randomized policies is unique to the time-inconsistent case. 

Abstract We consider three equilibrium concepts proposed in the literature for time‐inconsistent stopping problems, including mild equilibria (introduced in Huang and Nguyen‐Huu (2018)), weak equilibria (introduced in Christensen and Lindensjö (2018)), and strong equilibria (introduced in Bayraktar et al. (2021)). The discount function is assumed to be log subadditive and the underlying process is one‐dimensional diffusion. We first provide necessary and sufficient conditions for the characterization of weak equilibria. The smooth‐fit condition is obtained as a by‐product. Next, based on the characterization of weak equilibria, we show that an optimal mild equilibrium is also weak. Then we provide conditions under which a weak equilibrium is strong. We further show that an optimal mild equilibrium is also strong under a certain condition. Finally, we provide several examples including one showing a weak equilibrium may not be strong, and another one showing a strong equilibrium may not be optimal mild. 

A new definition of continuous-time equilibrium controls is introduced. As opposed to the standard definition, which involves a derivative-type operation, the new definition parallels how a discrete-time equilibrium is defined and allows for unambiguous economic interpretation. The terms “strong equilibria” and “weak equilibria” are coined for controls under the new and standard definitions, respectively. When the state process is a time-homogeneous continuous-time Markov chain, a careful asymptotic analysis gives complete characterizations of weak and strong equilibria. Thanks to the Kakutani–Fan fixed-point theorem, the general existence of weak and strong equilibria is also established under an additional compactness assumption. Our theoretic results are applied to a two-state model under nonexponential discounting. In particular, we demonstrate explicitly that there can be incentive to deviate from a weak equilibrium, which justifies the need for strong equilibria. Our analysis also provides new results for the existence and characterization of discrete-time equilibria under infinite horizon.